Pre/Post Comparison of Medical Students' Self-Reported Competence Ratings for Content and Skill Areas Included in an End of Third Year Assessment

BACKGROUND The End of Third Year Assessment (EOTYA) evaluates medical students using multiple Objective Structured Clinical Examinations (OSCE) and offers comprehensive feedback to students on their content knowledge and skills. This study measures the change in students' perceived level of competence before and after completing the EOTYA. RESULTS A total of ninety-one students (Class of 2004) completed both the pre-and post-questionnaires. The initial paired t-test comparison indicated a statistically significant difference between the pre-and post-items, which were clustered into content and skill areas (see Table 1). The results of the pre/post comparison indicated a statistically significant shift in the proportion of " positive " student ratings in three content areas (obstetrics/gynecology, psychiatry, and surgery; z=3.58-4.37, p<.05) and in one skill area (counseling/patient education, z=2.31, p<.05). Tables 2 and 3 and Figures 1 and 2 outlines these changes. Table 4 contains the correlation analysis between the post-competence ratings and student performance in selected content and skill areas, which reflected a significant positive association. METHOD A self-reported evaluation instrument using a 5-point Likert scale was developed for this study. A comparison of students' pre-and post-competence ratings for each of six EOTYA content areas (family medicine, medicine, obstetrics-gynecology, pediatrics, psychiatry, and surgery) and each of six EOTYA skills areas (history taking, physical examination, interviewing, professionalism, counseling and education, and problem solving) was conducted. The analysis compared the pre-and post-difference in the percentage of " positive " ratings using an approximation to the binomial distribution. In addition, items were clustered into content and skill areas, then a paired t-test analysis compared pre-and post-clusters for statistically significant differences. A subsequent correlation analysis was conducted between the post-competence ratings and actual student scores in selected EOTYA skill and content areas. CONCLUSION This study indicated students' perceived level of competence improved after completing the EOTYA. These results could offer another important element of summative feedback to third year medical students. GOAL To develop additional summative feedback for third year medical students emanating from the EOTYA and a newly developed self-assessment tool.